Control device, control method, unmanned aircraft, information processing device, information processing method, and program

ABSTRACT

The present technology relates to a control device, a control method, an unmanned aircraft, an information processing device, an information processing method, and a program capable of reflecting the user&#39;s intentions and reducing damage in the event of a failure in an unmanned aircraft. A control device of a first aspect of the present technology is a device that controls movement of an unmanned aircraft during a fall according to a control command generated for an image showing a falling position, captured by the unmanned aircraft. The present technology can be applied to a device that controls a drone that controls movement so as to shift the falling position when an aircraft fails.

TECHNICAL FIELD

The present technology particularly relates to a control device, acontrol method, an unmanned aircraft, an information processing device,an information processing method, and a program capable of reflectingthe user's intentions and reducing damage in the event of a failure inan unmanned aircraft.

BACKGROUND ART

In recent years, a small unmanned aircraft called a drone, which can beremotely controlled, has been attracting attention. If an unmannedaircraft fails during flight and becomes uncontrollable, it may fall andlead to a major accident. Therefore, various technologies for reducingdamage in the event of a failure have been proposed.

For example, PTL 1 proposes a technique for estimating a fall range anddetecting a person on the basis of an image obtained by photographingthe lower part of the aircraft, and controlling the aircraft so that theposition of the person and the fall range do not overlap.

CITATION LIST Patent Literature

[PTL 1]

WO 2017/033976

SUMMARY Technical Problem

In the technique disclosed in PTL 1, it is necessary to determine thedetection target in advance. Further, since the control of the aircraftis entrusted to the self-sustaining control, it is not possible toreflect the values of the user who operates the unmanned aircraft andthe intention of the user according to the situation.

The present technology has been made in view of such a situation, andreflects the intention of the user so that the damage in the event of afailure of the unmanned aircraft can be reduced.

Solution to Problem

A control device according to a first aspect of the present technologyincludes a control unit that controls movement of an unmanned aircraftduring a fall according to a control command generated for an imageshowing a falling position, captured by the unmanned aircraft.

An unmanned aircraft according to a second aspect of the presenttechnology includes: an imaging unit that captures a surroundingsituation; and a control unit that controls movement of an unmannedaircraft during a fall according to a control command generated for animage showing a falling position, captured by the imaging unit.

An information processing device according to a third aspect of thepresent technology includes: a display control unit that displays animage showing a falling position, captured by an unmanned aircraft; ageneration unit that generates a control command used for controllingthe movement of the unmanned aircraft with respect to the image; and atransmitting unit that transmits the control command to the unmannedaircraft.

In the first aspect of the present technology, the movement of anunmanned aircraft during a fall is controlled according to a controlcommand generated for an image showing a falling position, captured bythe unmanned aircraft.

In the second aspect of the present technology, a surrounding situationis captured, and the movement of an unmanned aircraft during a fall iscontrolled according to a control command generated for an image showinga falling position.

In the third aspect of the present technology, an image showing afalling position, captured by an unmanned aircraft is displayed, acontrol command used for controlling the movement of the unmannedaircraft with respect to the image is generated, and the control commandis transmitted to the unmanned aircraft.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram showing a configuration example of a control systemaccording to an embodiment of the present technology.

FIG. 2 is a diagram showing a state at the time of falling.

FIG. 3 is a diagram showing a display example of a composite image.

FIG. 4 is a diagram showing an example of user operation.

FIG. 5 is a diagram showing a relationship between an image captured byan unmanned aircraft and a composite image displayed on a smartphone.

FIG. 6 is a diagram showing another configuration example of acontroller.

FIG. 7 is a block diagram showing a configuration example of an unmannedaircraft.

FIG. 8 is a diagram showing an example of a method of estimating afalling position.

FIG. 9 is a diagram showing an example of images used for generating acomposite image.

FIG. 10 is a diagram showing an example of a method of synthesizingcaptured images.

FIG. 11 is a diagram showing an example of a direction on a compositeimage coordinate system represented by a control command.

FIG. 12 is a diagram showing an example of controlling the movement ofan unmanned aircraft that has received a control command.

FIG. 13 is a diagram showing an example of the direction of rotation ofa motor in a falling mode.

FIG. 14 is a diagram showing a configuration example of a controller.

FIG. 15 is a flowchart illustrating a fall damage mitigation process foran unmanned aircraft.

FIG. 16 is a flowchart illustrating a composite image display process ofthe controller.

FIG. 17 is a diagram showing an example of a user's operation fordesignating a falling position.

FIG. 18 is a diagram showing an example of the movement of an unmannedaircraft that has received a control command.

FIG. 19 is a block diagram showing a configuration example of anunmanned aircraft that receives a control command for designating afalling position.

FIG. 20 is a diagram showing an example of objects in the vicinity of afalling position of an unmanned aircraft.

FIG. 21 is a diagram showing a display example of a composite image inwhich object detection results are synthesized.

FIG. 22 is a block diagram showing a configuration example of anunmanned aircraft that detects an object appearing in a composite image.

FIG. 23 is a flowchart illustrating a fall damage mitigation processperformed by an unmanned aircraft.

FIG. 24 is a diagram showing an example of actions for avoiding acollision.

FIG. 25 is a diagram showing a display example of a composite imageshowing a direction of movement according to a plan.

FIG. 26 is a diagram showing an example of a user's operationinstructing to move in a direction different from the planned direction.

FIG. 27 is a diagram showing an example of movement of an unmannedaircraft.

FIG. 28 is a block diagram showing a configuration example of anunmanned aircraft having an autonomous avoidance function.

FIG. 29 is a flowchart illustrating a fall damage mitigation processperformed by an unmanned aircraft.

FIG. 30 is a diagram showing a configuration example of a smartphone.

FIG. 31 is a block diagram showing a configuration example of computerhardware.

DESCRIPTION OF EMBODIMENTS

Hereinafter, embodiments of the present technology will be described.Note that the description will be given in the following order.

-   1. Example of first control of unmanned aircraft-   2. Configuration of each device-   3. Operation of each device-   4. Example of second control of unmanned aircraft-   5. Example of third control of unmanned aircraft-   6. Example of fourth control of unmanned aircraft-   7. Modification

<Example of First Control of Unmanned Aircraft>

FIG. 1 is a diagram showing a configuration example of a control systemaccording to an embodiment of the present technology.

The control system of FIG. 1 includes an unmanned aircraft 1 and acontroller 2.

The unmanned aircraft 1 is a so-called drone, and moves (flies)according to a signal from the controller 2. The unmanned aircraft 1 maybe an autonomously moving aircraft. In FIG. 1, the unmanned aircraft 1is moving over a house O1 and a car O2 parked next to the house O1.

The unmanned aircraft 1 is an aircraft equipped with a camera. The imagecaptured while the unmanned aircraft 1 is moving is transmitted to thecontroller 2 by wireless communication as indicated by a broken linearrow.

The image transmitted from the unmanned aircraft 1 may be a moving imageor a still image. Images may be transmitted by wired communicationinstead of wireless communication.

The controller 2, which is the transmitter of the unmanned aircraft 1,receives the image transmitted from the unmanned aircraft 1 and displaysthe same on a smartphone 3. In the example of FIG. 1, the controller 2is a device that uses the display of the smartphone 3 attached to thehousing of the controller 2 as the display destination of the imagecaptured by the camera of the unmanned aircraft 1.

In this way, the user can operate the controller 2 while looking at theimage displayed on the smartphone 3 and control the unmanned aircraft 1even when the unmanned aircraft 1 is moving away from the user.

By the way, an accident such as a propeller coming off or a motorfailure may occur. In this case, the unmanned aircraft 1 will fall.

In the control system of FIG. 1, imaging is continued even while theunmanned aircraft 1 is falling, and the image during the fall ispresented to the user. The user can control the movement of the unmannedaircraft 1 by looking at the image during the fall.

FIG. 2 is a diagram showing a state at the time of falling.

When a part of the aircraft fails, the unmanned aircraft 1 falls whilerotating, for example, as indicated by a dotted arrow A1.

At the time of falling, the unmanned aircraft 1 synthesizes informationindicating the falling position with the image captured by the camera,and transmits a composite image to the controller 2.

The smartphone 3 attached to the controller 2 displays the compositeimage transmitted from the unmanned aircraft 1.

Since the information indicating the falling position is displayed, theuser can move the unmanned aircraft 1 in the direction of avoiding thehouse O1 and the car O2 while looking at the composite image displayedon the smartphone 3. In the example of FIG. 2, the unmanned aircraft 1avoids the house O1 and the car O2 as indicated by a solid arrow A3 inresponse to the user performing an operation such as moving the unmannedaircraft 1 in the direction indicated by a blank arrow A2.

FIG. 3 is a diagram showing a display example of a composite image.

As shown in FIG. 3, the display 3A of the smartphone 3 displays acomposite image showing the falling position of the unmanned aircraft 1.The falling position of the unmanned aircraft 1 is an estimated positionin consideration of surrounding environmental information such as windspeed.

For example, the falling position of the unmanned aircraft 1 isrepresented by a falling position image P. In the example of FIG. 3, thefalling position image P is an image in which a cross is arranged in acircle. The center of the cross that constitutes the falling positionimage P represents the falling position. The falling position may berepresented by a point or a region.

In the example of FIG. 3, the falling position is the position on thehouse O1. The car O2 is on the right side of the house O1. If theunmanned aircraft 1 falls as it is, the unmanned aircraft 1 will collidewith the house O1.

Since the falling position and an object such as the house O1 aredisplayed, the user can indicate the direction of movement of theunmanned aircraft 1 in the direction of avoiding the object with whichthe unmanned aircraft is expected to collide or the direction in whichdamage is reduced even if the unmanned aircraft 1 collides with theobject while looking at the composite image. In the example of FIG. 3,the left direction on the composite image is the direction in which theobject is considered to be avoided or the damage is reduced.

FIG. 4 is a diagram showing an example of user operation.

By looking at the display of the composite image, as shown in FIG. 4,the user tilts the stick of the controller 2 to the left and instructsthe left direction in the composite image, that is, the direction ofavoiding the house O1 and the car O2 as the movement direction. In FIG.4, the hatched arrow on the composite image shown indicates thedirection in which the unmanned aircraft 1 is desired to be moved, andis not an image displayed superimposed on the composite image.

The image captured by the unmanned aircraft 1 at the time of falling isan image in which the imaging range keeps changing at a high speed. Forthis reason, it is difficult for the user to confirm the position wherethe unmanned aircraft 1 may fall or the object with which the unmannedaircraft 1 may collide just by looking at the captured image.

By displaying the composite image in which the falling position image Pis displayed, the user can allow the unmanned aircraft 1 to fall at aposition where the damage at the time of the fall is considered to bereduced according to the user's values and the situation at the time ofthe fall.

In addition, it is possible to avoid a collision with an object that isdifficult to detect by object detection or the like.

The user can designate the direction in which the unmanned aircraft 1 ismoved with reference to the direction in the composite image during thefall of the unmanned aircraft 1. The direction designated by the user isconverted into a direction in the coordinate system of the unmannedaircraft 1 according to the attitude of the unmanned aircraft 1 at thattime, and the moving direction is controlled.

FIG. 5 is a diagram showing the relationship between the image capturedby the unmanned aircraft 1 and the composite image displayed on thesmartphone 3.

As shown on the left side of FIG. 5, when a failure occurs in theaircraft, the unmanned aircraft 1 falls while rotating. FIG. 5 shows thestate of the unmanned aircraft 1 at each time of times T1 to T7. Adotted arrow A11 indicating the direction directly below indicates thefalling direction when there is no user operation.

Blank triangles shown on the bottom surface side of the unmannedaircraft 1 represent the angle of view of the camera provided on theunmanned aircraft 1. For example, the direction of the angle of view ofthe camera at time T1 is the direction directly below.

In this case, as shown at the tip of arrow #1, the angle of view of thecamera overlaps with the display range of the composite image as awhole.

A rectangular frame F shown on the right side of FIG. 5 represents thedisplay range of the composite image (the range displayed on the display3A). The display range of the composite image is set so that, forexample, the falling position is at the center. The hatched rangerepresents the range of the angle of view of the camera.

At time T1, the composite image is displayed using the image captured bythe unmanned aircraft 1 as it is.

The direction of the angle of view of the camera at time T2 isdiagonally downward to the right. In this case, as shown at the tip ofarrow #2, the angle of view of the camera partially overlaps with thedisplay range of the composite image.

At time T2, the composite image is displayed using a part of the imagecaptured by the unmanned aircraft 1 that overlaps with the display rangeof the composite image. Of the entire composite image, a range otherthan the range displayed using the image captured at time T2 isdisplayed using, for example, the images captured at times before timeT2.

The direction of the angle of view of the camera at time T3 isdiagonally upward to the right. In this case, as shown at the tip ofarrow #3, the angle of view of the camera does not overlap with thedisplay range of the composite image.

At time T3, the composite image is displayed using the images capturedat times before time T3.

If the angle of view of the camera and the display range of thecomposite image do not match, the composite image may not be displayed.

The angles of view of the camera at times T4 and T5 do not overlap withthe display range of the composite image, as shown at the tip of each ofarrows #4 and #5, like the angle of view of the camera at time T3.

At times T4 and T5, the composite image is displayed using the imagescaptured at the times before the times T4 and T5, respectively.

The angle of view of the camera at time T6 partially overlaps with thedisplay range of the composite image, as shown at the tip of arrow #6.

At time T6, the composite image is displayed using a part of the imagecaptured by the unmanned aircraft 1 that overlaps with the display rangeof the composite image.

The angle of view of the camera at time T7 overlaps with the displayrange of the composite image as a whole, as shown at the tip of arrow#7.

At time T7, the composite image is displayed using the image captured bythe unmanned aircraft 1 as it is.

In this way, the display 3A of the smartphone 3 viewed by the usercontinues to display only the range including the falling position amongthe images captured during the fall. In this way, the user can instructthe moving direction of the unmanned aircraft 1 while looking at thecomposite image showing the falling position even when the unmannedaircraft 1 is falling while rotating.

In the above, it is assumed that the controller 2 is a controller inwhich the smartphone 3 is attached, but it may be a controller inanother form.

FIG. 6 is a diagram showing another configuration example of thecontroller 2.

The controller 2 shown in A of FIG. 6 is a controller in which a display2A is provided in a housing. The user can steer the unmanned aircraft 1by looking at the composite image displayed on the display 2A and thelike. For example, the direction of movement is indicated by thedirection in which the user tilts the stick, and the amount of movementis indicated by the amount of tilting the stick.

As shown in B of FIG. 6, the smartphone 3 itself can be used as atransmitter. The user can steer the unmanned aircraft 1 by performing aswipe operation or the like by looking at the composite image displayedon the display 3A or the like.

<Configuration of Each Device>

Configuration of Unmanned Aircraft 1

FIG. 7 is a block diagram showing a configuration example of theunmanned aircraft 1.

As shown in FIG. 7, the unmanned aircraft 1 includes a sensor 11 and aninformation processing unit 12.

The sensor 11 includes a wind speed sensor 21, an imaging sensor 22, aposition sensor 23, and an IMU (Inertial Measurement Unit) 24. Theimaging sensor 22 is provided in the camera mounted on the unmannedaircraft 1.

The wind speed sensor 21 detects and outputs a wind speed vectorincluding a wind direction and a wind speed (air volume).

The imaging sensor 22 is configured of an image sensor, a stereo camera,a ToF (Time of Flight) sensor, a LiDER (Light Detection and Ranging,Laser Imaging Detection and Ranging), and the like.

The image sensor constituting the imaging sensor 22 images thesurrounding situation and outputs image data.

Further, for example, a stereo camera constituting the imaging sensor 22calculates the distance to each object captured in the image on thebasis of the image obtained by imaging, and outputs the distanceinformation. The distance to the object may be detected by a ToF sensoror the like.

The position sensor 23 is configured of a GPS (Global PositioningSystem) sensor, a barometer, and the like. The position sensor 23receives radio waves from the satellite, performs positioning, andoutputs position information of the unmanned aircraft 1.

The IMU 24 includes an acceleration sensor, a gyro sensor, a magneticsensor, and the like. The IMU 24 measures the speed, acceleration,magnetic field strength, and the like of the unmanned aircraft 1 andoutputs it as IMU information.

The information processing unit 12 includes a wind speed vectoracquisition unit 31, an image acquisition unit 32, a positioninformation acquisition unit 33, an IMU information acquisition unit 34,an internal state acquisition unit 35, a self-position and motionestimation unit 36, a falling position estimation unit 37, and a failureand fall determination unit 38, an image synthesis unit 39, a datatransmitting unit 40, a data receiving unit 41, and an aircraft controlunit 42.

The wind speed vector acquisition unit 31 acquires the wind speed vectoroutput from the wind speed sensor 21, and outputs the wind speedinformation representing the acquired wind speed vector to the fallingposition estimation unit 37.

The image acquisition unit 32 acquires the image data and the distanceinformation output from the imaging sensor 22, and outputs the same tothe self-position and motion estimation unit 36. The image acquisitionunit 32 outputs the image data to the image synthesis unit 39.

The position information acquisition unit 33 acquires the positioninformation output from the position sensor 23 and outputs the same tothe self-position and motion estimation unit 36.

The IMU information acquisition unit 34 acquires the IMU informationoutput from the IMU 24 and outputs the same to the self-position andmotion estimation unit 36.

The internal state acquisition unit 35 acquires the output values of acurrent monitor, a voltage monitor, an encoder, and the like of theunmanned aircraft 1 as information indicating the internal state, andoutputs the output values to the failure and fall determination unit 38.

The self-position and motion estimation unit 36 calculates the flightstate of the unmanned aircraft 1 on the basis of the image data anddistance information supplied from the image acquisition unit 32, theposition information supplied from the position information acquisitionunit 33, and the IMU information supplied from the IMU informationacquisition unit 34. The flight state includes the position, attitude,speed, angular velocity, acceleration, angular acceleration, and thelike of the aircraft of the unmanned aircraft 1.

The self-position and motion estimation unit 36 estimates the inertialforce and gravity applied to the unmanned aircraft 1 on the basis of theflight state.

The self-position and motion estimation unit 36 outputs informationrepresenting the flight state and the inertial force and gravity appliedto the unmanned aircraft 1 as the self-position and motion estimationresult. The self-position and motion estimation result output from theself-position and motion estimation unit 36 is supplied to the fallingposition estimation unit 37, the failure and fall determination unit 38,the image synthesis unit 39, and the aircraft control unit 42.

The falling position estimation unit 37 estimates the falling positionof the falling unmanned aircraft 1 on the basis of the wind speedinformation supplied from the wind speed vector acquisition unit 31 andthe self-position and motion estimation result supplied from theself-position and motion estimation unit 36.

FIG. 8 is a diagram showing an example of a method of estimating thefalling position.

As indicated by blank arrows in FIG. 8, an inertial force in a directioncorresponding to the moving direction up to that point is applied to theunmanned aircraft 1 that has failed during flight, and a downwardgravity is applied. In addition, wind force corresponding to the winddirection is applied. The inertial force, gravity, and wind forceapplied to an unmanned aircraft are collectively referred to as externalforce.

The falling position estimation unit 37 estimates the falling positionon the basis of the flight state and the external force. In FIG. 8, theposition C1 which is a position deviated from directly below theposition where the failure occurred is estimated as the fallingposition.

The falling position estimation unit 37 sets a region having apredetermined shape centered on the estimated falling position as theestimated falling position range. The estimated falling position rangeis set as a region that gradually narrows as the unmanned aircraft 1approaches the ground.

Returning to the description of FIG. 7, the falling position estimationunit 37 outputs the falling position estimation result representing theestimated falling position and the estimated falling position range tothe image synthesis unit 39.

The failure and fall determination unit 38 determines the failure orfall of the aircraft of the unmanned aircraft 1 on the basis of theinformation representing the internal state supplied from the internalstate acquisition unit 35 and the self-position and motion estimationresult supplied from the self-position and motion estimation unit 36.

Specifically, the failure and fall determination unit 38 diagnoses afailure such as a failure of the motor of the unmanned aircraft 1,damage to the propeller, and wrapping of a foreign object using theinternal state. The failure and fall determination unit 38 diagnoses afailure on the basis of the amount of deviation between the actualinternal state and the internal state assumed when there is no failure.

In this way, the failure and fall determination unit 38 detects that afailure that hinders the movement has occurred and the unmanned aircraft1 cannot move as expected and begins to fall. For the determination offailure and fall, a rule-based determination may be performed, or amodel obtained by machine learning may be used.

The failure and fall determination unit 38 outputs the failure and falldetermination information to the image synthesis unit 39 and theaircraft control unit 42. The failure and fall determination informationincludes, for example, information indicating whether a failure hasoccurred and information indicating a failed portion of the unmannedaircraft 1.

The image synthesis unit 39 generates a composite image by synthesizingthe falling position image with the image in which the falling positionis captured. For the generation of the composite image, the image datasupplied from the image acquisition unit 32, the self-position andmotion estimation result supplied from the self-position and motionestimation unit 36, the falling position estimation result supplied fromthe falling position estimation unit 37, and the failure and falldetermination information supplied from the failure and falldetermination unit 38 are used.

FIG. 9 is a diagram showing an example of images used for generating acomposite image.

FIG. 9 shows the state of the unmanned aircraft 1 at each time of timesT1 to T3. Here, it is assumed that the unmanned aircraft 1 is providedwith one camera. The range indicated by a broken line represents therange of the angle of view of the camera at each time.

At times T1 to T3, images are captured in a state where the camera isdirected downward, diagonally downward to the left, and further to theleft than the direction at time T2, respectively, and the capturedimages P11 to P13 having the angles of view indicated by the trapezoidsin FIG. 9 are acquired.

For example, at time T3, a composite image is generated on the basis ofthe captured images P11 to P13 captured in this way.

In the example of FIG. 9, at time T3, the estimated falling positionrange represented by an ellipse is estimated. Among the captured imagesP11 to P13, the region in which the estimated falling position range andthe position directly below the unmanned aircraft 1 at time T3 arecaptured is used for generating the composite image.

FIG. 10 is a diagram showing an example of a method of synthesizingcaptured images.

When the captured images P 11 to P13 are viewed from directly above, theangle of view of each image is represented as a shape as shown in FIG.10. FIG. 10 shows the shape of each image when viewed from directlyabove with respect to a plane representing the ground.

In the image synthesis unit 39, the captured images P11 to P13 areprojected and converted with respect to the plane representing theground using the position and attitude of the unmanned aircraft 1 andthe internal and external parameters of the camera.

In the image synthesis unit 39, the captured images P11 to P13 after theprojective conversion are synthesized so as to be stitched together, anda range including the position corresponding to the position directlybelow at time T3 and the estimated falling position range is cut out. Inthe example of FIG. 10, an image showing the range indicated by a brokenline is cut out as a cut-out image P21.

In the image synthesis unit 39, a composite image is generated bysynthesizing a falling position image representing the estimated fallingposition range and the position corresponding to the position directlybelow on the cut-out image P21. The composite image generated in thisway is output from the image synthesis unit 39 to the data transmittingunit 40 of FIG. 7.

When a plurality of cameras is provided in the unmanned aircraft 1, thecaptured images in which the estimated falling position range and theangle of view overlap are used for generating the composite image.

The data transmitting unit 40 transmits the composite image suppliedfrom the image synthesis unit 39 to the controller 2. The compositeimage transmitted by the data transmitting unit 40 is displayed on thedisplay 3A of the smartphone 3 and is used for instructing the directionof movement of the unmanned aircraft 1.

The data receiving unit 41 receives a control command representing thecontent of the user's operation transmitted from the controller 2 andoutputs the control command to the aircraft control unit 42. The controlcommand indicates, for example, the direction instructed by the user whoviewed the composite image.

The aircraft control unit 42 determines whether a failure has occurredon the basis of the failure and fall determination information suppliedfrom the failure and fall determination unit 38, and sets an operationmode.

The operation mode of the unmanned aircraft 1 includes an in-flight modeand a falling mode. The in-flight mode is an operation mode set when afailure has not occurred, and the falling mode is an operation mode setwhen a failure has occurred.

The aircraft control unit 42 controls the movement of the unmannedaircraft 1 in response to the control command supplied from the datareceiving unit 41.

When the operation mode is the in-flight mode, the aircraft control unit42 controls the position and attitude in an aircraft coordinate system.

The aircraft coordinate system represents the coordinate system in theunmanned aircraft 1. When the operation mode is the in-flight mode, theuser who operates the controller 2 performs an operation on the aircraftcoordinate system to control the movement of the unmanned aircraft 1.

On the other hand, when the operation mode is the falling mode, theaircraft control unit 42 performs control in consideration of thefailure location on the basis of the self-position and motion estimationresult supplied from the self-position and motion estimation unit 36 andthe failure and fall determination information supplied from the failureand fall determination unit 38.

In this case, the aircraft control unit 42 converts the direction of theuser's instruction represented by a composite image coordinate systeminto the direction of the aircraft coordinate system and performscontrol. Since the direction of the user's instruction represented bythe control command is the direction in which the composite image isviewed, it is represented as the direction of the composite imagecoordinate system, which is the coordinate system in the compositeimage.

That is, when the operation mode is the falling mode, the user whooperates the controller 2 performs an operation on the composite imagecoordinate system to control the movement of the unmanned aircraft 1.

FIG. 11 is a diagram showing an example of a direction on the compositeimage coordinate system represented by a control command.

When an object to avoid a collision exists within the estimated fallingposition range, transmits a control command instructing the movement tothe left with respect to the composite image to the unmanned aircraft 1by an operation such as tilting the stick of the controller 2 to theleft. At this time, as indicated by arrow Al2 in FIG. 11, a controlcommand instructing the movement to the left on the composite imagecoordinate system is transmitted to the unmanned aircraft 1.

By instructing the movement in the depth direction in the compositeimage, it is possible to perform control so as to accelerate the fall.

FIG. 12 is a diagram showing an example of controlling the movement ofthe unmanned aircraft 1 that has received the control command.

When the movement to the left as described with reference to FIG. 11 isinstructed on the composite image, the aircraft control unit 42 controlsthe aircraft so that the actual falling position is located to the leftof the estimated falling position range as indicated by arrow A13 inFIG. 12.

Specifically, the aircraft control unit 42 converts the directiondesignated by the user into the direction on the aircraft coordinatesystem at time T3 using a predetermined conversion matrix and controlsthe aircraft.

When it is not possible to move in any direction due to a motor failureor the like, the aircraft control unit 42 considers the position andattitude of the aircraft, and controls the direction of rotation of anoperable motor so that thrust is generated in the direction designatedby the user.

FIG. 13 is a diagram showing an example of the direction of rotation ofthe motor in the falling mode.

FIG. 13 shows the states of the unmanned aircraft 1 that falls whilerotating at each time of times T1 to T9. Further, as indicated by ablank arrow, it is assumed that the user who has viewed the compositeimage has designated the movement to the left.

When there is only one operable motor, the aircraft control unit 42rotates the motor in a direction in which the inner product of thethrust vector generated when the motor is rotated and the directionvector pointing in the direction designated by the user becomespositive.

In FIG. 13, it is assumed that, among two propeller motors, the leftpropeller motor fails with reference to, for example, the direction attime T1 in which the vertical direction is correct, and only the rightpropeller motor can operate. A solid arrow shown near the motor of thepropeller on the right side represents the thrust vector generated bythe forward rotation of the motor. A dotted arrow represents the thrustvector generated by the reverse rotation of the motor. Here, the forwardrotation represents rotation in a direction in which buoyancy isgenerated in a normal state.

At time T1 when the upper surface of the aircraft is facing straight up,the internal product of the thrust vector generated by rotating themotor and the direction vector facing in the direction designated by theuser becomes 0, so the aircraft control unit 42 does not rotate themotor.

On the other hand, at times T2 to T4 when the upper surface of theaircraft is facing to the left, the aircraft control unit 42 rotates themotor in the forward direction so as to generate a thrust vector whoseinner product with the direction vector pointing in the directiondesignated by the user is positive. In this way, the unmanned aircraft 1will fall while shifting to the left.

At time T5 when the upper surface of the aircraft is facing directlybelow, the internal product of the thrust vector generated by rotatingthe motor and the direction vector facing in the direction designated bythe user becomes 0, so the aircraft control unit 42 does not rotate themotor.

At times T6 to T8 when the upper surface of the aircraft is facing tothe right, the aircraft control unit 42 rotates the motor in the reversedirection so as to generate a thrust vector whose inner product with thedirection vector pointing in the direction designated by the user ispositive. In this way, the unmanned aircraft 1 will fall while shiftingto the left.

At time T9 when the upper surface of the aircraft is facing straight up,the internal product of the thrust vector generated by rotating themotor and the direction vector pointing in the direction designated bythe user is 0, so the aircraft control unit 42 does not rotate themotor.

As described above, the aircraft control unit 42 can shift the fallingposition to the left by controlling the direction of rotation of themotor that can operate so as to generate a thrust to the left designatedby the user.

Configuration of Controller 2

FIG. 14 is a diagram showing a configuration example of the controller2. The smartphone 3 is connected to the controller 2 via wired orwireless communication.

As shown in FIG. 14, the controller 2 includes an information processingunit 51 and an input unit 52.

The information processing unit 51 includes a data receiving unit 61, adata display control unit 62, an input acquisition unit 63, and a datatransmitting unit 64.

The data receiving unit 61 receives the composite image transmitted fromthe unmanned aircraft 1 and outputs the same to the data display controlunit 62.

The data display control unit 62 outputs the composite image suppliedfrom the data receiving unit 61 to the display 3A of the smartp hone 3and displays the same.

The input acquisition unit 63 acquires the instruction informationoutput from the input unit 52 and outputs the instruction information tothe data transmitting unit 64. The instruction information representsthe direction and the amount of movement designated by the user.

The data transmitting unit 64 transmits the instruction informationsupplied from the input acquisition unit 63 to the unmanned aircraft 1as a control command.

The input unit 52 is configured of a stick, a touch panel, or the like.The input unit 52 detects the user's operation and outputs instructioninformation according to the detected user's operation.

<Operation of Each Device>

Here, the operation of each device having the above-describedconfiguration will be described.

Operation of Unmanned Aircraft 1

First, the fall damage mitigation process of the unmanned aircraft 1will be described with reference to the flowchart of FIG. 15.

The fall damage mitigation process of FIG. 15 is started, for example,when the flight of the unmanned aircraft 1 is started. The operationmode of the unmanned aircraft 1 at the start of processing is thein-flight mode.

In step S1, the information processing unit 12 acquires the sensor datasupplied from the sensor 11. Specifically, the wind speed vectoracquisition unit 31, the image acquisition unit 32, the positioninformation acquisition unit 33, and the IMU information acquisitionunit 34 acquire wind speed information, image data and distanceinformation, position information, and IMU information, respectively.

In step S2, the self-position and motion estimation unit 36 estimatesthe self-position and motion estimation result on the basis of the imagedata, the distance information, the position information, and the IMUinformation.

In step S3, the failure and fall determination unit 38 determineswhether the aircraft of the unmanned aircraft 1 has failed or fallen onthe basis of the information indicating the internal state and theself-position and motion estimation result.

If it is determined in step S3 that the aircraft of the unmannedaircraft 1 has failed or fallen, the process proceeds to step S4.

In step S4, the falling position estimation unit 37 estimates thefalling position of the unmanned aircraft 1 during the fall on the basisof the wind speed information and the self-position and motionestimation result.

In step S5, the image synthesis unit 39 generates a composite image bysynthesizing the falling position image with the image showing thefalling position on the basis of the image data, the self-position andmotion estimation result, the falling position estimation result, andthe failure and fall determination information.

In step S6, the aircraft control unit 42 sets the operation mode to thefalling mode on the basis of the failure and fall determinationinformation.

In step S7, the data transmitting unit 40 transmits the composite imageto the controller 2.

If the composite image is transmitted to the controller 2 in step S7, orif it is determined in step S3 that the aircraft of the unmannedaircraft 1 has neither failed nor fallen, the process proceeds to stepS8.

In step S8, the data receiving unit 41 determines whether the controlcommand has been received from the controller 2.

If it is determined in step S8 that the control command has not beenreceived, the process returns to step S1 and the subsequent processingis performed.

On the other hand, if it is determined in step S8 that the controlcommand has been received, the process proceeds to step S9. A controlcommand is supplied from the data receiving unit 41 to the aircraftcontrol unit 42.

In step S9, the aircraft control unit 42 determines whether theoperation mode is the falling mode.

If it is determined in step S9 that the operation mode is the fallingmode, the process proceeds to step S10.

In step S10, the aircraft control unit 42 converts the directionrepresented by the control command from the direction in the compositeimage coordinate system to the direction in the aircraft coordinatesystem on the basis of the composite image.

In step S11, the aircraft control unit 42 controls the motor of theunmanned aircraft 1 in consideration of the failure location on thebasis of the self-position and motion estimation result and the failureand fall determination information, and moves the unmanned aircraft 1 inthe desired direction corresponding to the control command.

On the other hand, if it is determined in step S9 that the operationmode is not the falling mode, the process proceeds to step S12.

In step S12, the aircraft control unit 42 controls the motor of theunmanned aircraft 1, and moves the unmanned aircraft 1 in a desireddirection corresponding to the control command.

The above processing is repeated during the flight or the fall of theunmanned aircraft 1.

Operation of Controller 2

Next, the composite image display process of the controller 2 will bedescribed with reference to the flowchart of FIG. 16.

In step S21, the data receiving unit 61 of the controller 2 receives thecomposite image transmitted from the unmanned aircraft 1.

In step S22, the data display control unit 62 outputs the compositeimage to the display 3A of the smartphone 3 and displays the same.

In step S23, the input unit 52 receives the user's operation andgenerates instruction information.

In step S24, the input acquisition unit 63 acquires the instructioninformation.

In step S25, the data transmitting unit 64 transmits the instructioninformation as a control command to the unmanned aircraft 1.

By the above-described processing, the user can allow the unmannedaircraft 1 to fall at a position where damage at the time of falling isconsidered to be reduced according to the user's values and thesituation at the time of falling.

<Example of Second Control of Unmanned Aircraft>

The falling position may be designated by the user instead of the falldirection. In this case, the aircraft of the unmanned aircraft 1 iscontrolled so as to fall at a position designated by the user.

FIG. 17 is a diagram showing an example of a user's operation fordesignating a falling position.

As shown in FIG. 17, the user designates the falling position bytouching on the display 3A having a touch panel. In FIG. 17, theposition where there is no house O1 or car O2, which is indicated by ablank cross, is designated by the user.

In this case, the smartphone 3 functioning as a transmitter transmits acontrol command representing a position designated by the user to theunmanned aircraft 1.

FIG. 18 is a diagram showing an example of movement of the unmannedaircraft 1 that has received a control command.

The position C11 indicated by a blank cross on the left side of FIG. 18represents a position in an actual three-dimensional space correspondingto the position on the composite image designated by the user.

Upon receiving the control command, the unmanned aircraft 1 controls theaircraft so as to fall at the position C11 according to the controlcommand, as indicated by the broken line.

FIG. 19 is a block diagram showing a configuration example of theunmanned aircraft 1 that receives a control command for designating afalling position.

The configuration of the unmanned aircraft 1 shown in FIG. 19 is thesame as the configuration described with reference to FIG. 7, exceptthat the falling position estimation unit 37 and the aircraft controlunit 42 are connected. Duplicate explanation will be omitted asappropriate.

The aircraft control unit 42 of FIG. 19 is supplied with the sameinformation as the falling position estimation result supplied to theimage synthesis unit 39 from the falling position estimation unit 37.

The aircraft control unit 42 calculates the falling position in thethree-dimensional space designated by the user on the basis of thecontrol command for designating the falling position supplied from thedata receiving unit 61. The aircraft control unit 42 performs feedbackcontrol on the basis of the difference between the falling positiondesignated by the user and the falling position estimation result, andmoves the aircraft of the unmanned aircraft 1 to the falling positiondesignated by the user.

As described above, the user can allow the unmanned aircraft 1 to fallat a designated position according to the user's values and situation.For example, when it is desired that the unmanned aircraft 1 fallsbetween the objects shown in the composite image, the user can allow theunmanned aircraft 1 to fall at such a desired position.

<Example of Third Control of Unmanned Aircraft>

The object detection result, which is the result of detecting the objectappearing in the composite image, may be synthesized with the compositeimage as information assisting the user.

FIG. 20 is a diagram showing an example of an object in the vicinity ofthe falling position of the unmanned aircraft 1.

In the example of FIG. 20, the falling position of the unmanned aircraft1 is the position on the house O1. The car O2 is parked next to thehouse O1, and a person O3 is standing next to the car O2.

The unmanned aircraft 1 detects an object on an image showing a fallingposition, and detects the house O1, the car O2, and the person O3.Specific objects such as a house, a car, and a person are detected.

The unmanned aircraft 1 generates a composite image in which pieces ofinformation representing the house O1, the car O2, and the person O3 aresynthesized, and transmits the composite image to the controller 2 todisplay the same on the smartphone 3.

FIG. 21 is a diagram showing a display example of a composite image inwhich the object detection results are synthesized.

As shown in FIG. 21, a composite image is displayed on the display 3A ofthe smartphone 3. In the composite image, the house O1, the car O2, andthe person O3 are photographed side by side.

In the composite image of FIG. 21, rectangular information R1 in whichL-shaped lines are synthesized is displayed so as to surround the houseO1. The rectangular information R1 represents a region on the compositeimage in which the house is detected. Above the rectangular informationR1, the character information of “House” indicating that there is ahouse is displayed.

Similarly, rectangular information R2 is displayed so as to surround thecar O2. The rectangular information R2 represents a region on thecomposite image in which the car is detected. Above the rectangularinformation R2, the character information of “Car” indicating that thereis a car is displayed.

Rectangular information R3 is displayed so as to surround the person O3.The rectangular information R3 represents a region on the compositeimage in which a person is detected. Above the rectangular informationR3, the character information of “Human” representing a person isdisplayed.

The user can recognize the type of object that the unmanned aircraft 1may collide with by looking at the rectangular information and thecharacter information displayed on the composite image.

Information indicating the moving direction recommended for the user maybe displayed on the composite image on the basis of the object detectionresult and the fall estimation range.

FIG. 22 is a block diagram showing a configuration example of theunmanned aircraft 1 that detects an object appearing in a compositeimage.

The configuration of the unmanned aircraft 1 shown in FIG. 22 is thesame as the configuration described with reference to FIG. 7, exceptthat an object detection unit 101 is provided. Duplicate explanationwill be omitted as appropriate.

The image synthesis unit 39 outputs an image in which the fallingposition is captured to the object detection unit 101. The objectdetection unit 101 is supplied with an image in which the fallingposition is captured, which is generated by the image synthesis unit 39as described with reference to FIG. 10, for example.

The image synthesis unit 39 generates a composite image by synthesizingrectangular information and character information together with thefalling position image with the image showing the falling position onthe basis of the object detection result supplied from the objectdetection unit 101.

The object detection unit 101 detects an object on the image suppliedfrom the image synthesis unit 39. The object detection unit 101 outputsthe object detection result to the image synthesis unit 39.

Here, with reference to the flowchart of FIG. 23, the fall damagemitigation process performed by the unmanned aircraft 1 having theabove-described configuration will be described.

The processes of steps S51 to S56 are the same as the processes of stepsS1 to S6 of FIG. 15, respectively. That is, when the aircraft of theunmanned aircraft 1 fails, an image showing the falling position issynthesized, and the operation mode of the unmanned aircraft 1 is set tothe falling mode.

In step S57, the object detection unit 101 detects an object on an imagein which the falling position is captured.

In step S58, the image synthesis unit 39 generates a composite image bysynthesizing the rectangular information and the character informationtogether with the falling position image with the image showing thefalling position on the basis of the object detection result.

The processes of steps S59 to S64 are the same as the processes of stepsS7 to S12 of FIG. 15, respectively. That is, the composite image istransmitted to the controller 2, and the movement of the unmannedaircraft 1 is controlled according to the operation by the user.

As described above, the user can recognize the type of object that theunmanned aircraft 1 may collide with, and can allow the unmannedaircraft 1 to fall at a position where the damage at the time of fallingis considered to be reduced according to the user's values and thesituation at the time of falling.

<Example of Fourth Control of Unmanned Aircraft>

An action plan may be made on the unmanned aircraft 1 on the basis of anobject detection result or the like, and the movement of the unmannedaircraft 1 may be autonomously controlled according to the plannedaction. For example, the actions required to avoid collisions withobjects in the composite image are planned.

FIG. 24 is a diagram showing an example of actions for avoiding acollision.

As shown in FIG. 24, when the position on the house O1 is the fallingposition, the unmanned aircraft 1 detects the house O1, the car O2, andthe person O3 on the basis of the image showing the falling position,and plans the actions necessary to avoid collisions with these objects.For example, as indicated by a hatched arrow, an action of allowing theaircraft to fall at a position in front of the house O1, the car O2, andthe person O3 is planned, and autonomous avoidance is performed.

FIG. 25 is a diagram showing a display example of a composite imageshowing the direction of movement according to the plan.

As shown in FIG. 25, an arrow indicating the direction of movementaccording to the plan is displayed in the composite image. The otherdisplay of the composite image shown in FIG. 25 is the same as thedisplay of the composite image described with reference to FIG. 21.

If the action planned by the unmanned aircraft 1 does not suit theuser's values and situation, the user who have viewed the compositeimage may instruct to move in a direction different from the plannedmovement direction as shown in FIG. 26.

In the example of FIG. 26, the direction of autonomous avoidance ispresented as the downward direction indicated by a hatched arrow,whereas the user instructs to move to the left. A control commandrepresenting the direction instructed by the user is transmitted to theunmanned aircraft 1.

FIG. 27 is a diagram showing an example of movement of the unmannedaircraft 1.

When a direction different from the direction of autonomous avoidance isdesignated by the user as described with reference to FIG. 26, theunmanned aircraft 1 prioritizes the user's instruction, as indicated byblank arrows in FIG. 27 and controls the aircraft so as to move in thedirection instructed by the user.

In this way, the user may be able to intervene in the autonomousavoidance by the unmanned aircraft 1.

Instead of simply giving priority to the user's instructions, thedirection of autonomous avoidance and the direction instructed by theuser may be combined to plan a new action.

In order to make it easier for the user to determine the direction ofmovement, information representing the direction in which the unmannedaircraft 1 can be easily moved may be displayed. For example, the winddirection and the air volume are displayed as information representingthe direction in which the unmanned aircraft 1 can be easily moved.

The time until the unmanned aircraft 1 falls may be displayed.

FIG. 28 is a block diagram showing a configuration example of theunmanned aircraft 1 having an autonomous avoidance function.

The configuration of the unmanned aircraft 1 shown in FIG. 28 is thesame as the configuration described with reference to FIG. 22 exceptthat an avoidance action generation unit 111 is provided. Duplicateexplanation will be omitted as appropriate.

The aircraft control unit 42 controls the aircraft of the unmannedaircraft 1 according to an action plan for autonomous avoidance suppliedfrom the avoidance action generation unit 111. When a control commandrepresenting a user's instruction is received, the aircraft control unit42 controls the aircraft of the unmanned aircraft 1 with priority givento the user's operation, as described above.

The object detection unit 101 calculates a three-dimensional position ofthe detected object in the image in which the falling position iscaptured. Distance information to the ground and distance information toan object are appropriately used for calculating the three-dimensionalposition. The object detection unit 101 outputs the informationrepresenting the three-dimensional position of the object and the objectdetection result to the avoidance action generation unit 111.

The self-position and motion estimation result is supplied to theavoidance action generation unit 111 from the self-position and motionestimation unit 36. Further, the avoidance action generation unit 111 issupplied with the falling position estimation result from the fallingposition estimation unit 37, and the failure and fall determinationinformation from the failure and fall determination unit 38.

The avoidance action generation unit 111 plans actions necessary foravoiding a collision with an object shown in the composite image on thebasis of the information supplied from each unit. Informationrepresenting the action planned by the avoidance action generation unit111 is supplied to the aircraft control unit 42.

When the moving object is shown in the composite image, the position ofthe moving object at the time when the unmanned aircraft 1 collides withthe ground or the object may be predicted by the avoidance actiongeneration unit 111. In this case, the action is planned using thepredicted position of the moving object.

Here, with reference to the flowchart of FIG. 29, the fall damagemitigation process performed by the unmanned aircraft 1 having theabove-described configuration will be described.

The processes of steps S101 to S109 are the same as the processes ofsteps S51 to S59 of FIG. 23, respectively. That is, the operation modeof the unmanned aircraft 1 is set to the falling mode, and the compositeimage is transmitted to the controller 2.

In step S110, the data receiving unit 41 determines whether the controlcommand has been received from the controller 2.

If it is determined in step S110 that the control command has not beenreceived, the process proceeds to step S111.

In step S111, the aircraft control unit 42 determines whether theoperation mode is the falling mode.

If it is determined in step S111 that the operation mode is not thefalling mode, the process returns to step S101, and subsequent processesare performed.

On the other hand, if it is determined in step S111 that the operationmode is the falling mode, the process proceeds to step S112.

In step S112, the avoidance action generation unit 111 plans an actionfor autonomous avoidance. After the action for autonomous avoidance isplanned, in step S115, the direction of movement of the unmannedaircraft 1 is controlled according to the planned action.

On the other hand, if it is determined in step S110 that the controlcommand has been received, the process proceeds to step S113.

The processes of steps S113 to S116 are the same as the processes ofsteps S61 to S64 of FIG. 23, respectively. That is, it is determinedwhether the operation mode is the falling mode, and the movement of theunmanned aircraft 1 is controlled according to the operation by the useraccording to the determination result.

As described above, the unmanned aircraft 1 can autonomously take anaction of avoiding a collision with an object shown in the compositeimage.

<Modification>

System Configuration

Although the sensor 11 and the information processing unit 12 areprovided in the unmanned aircraft 1 (FIG. 7), some functions of theinformation processing unit 12 may be realized in any device of thecontroller 2 or the smartphone 3.

FIG. 30 is a diagram showing a configuration example of the smartphone3.

As shown in FIG. 30, the information processing unit 151 is realized inthe smartphone 3. The configuration of the information processing unit151 shown in FIG. 30 is the same as that of the information processingunit 12 of the unmanned aircraft 1 described with reference to FIG. 7,except that a display unit 161, an input acquisition unit 162, a controlcommand generation unit 163, and a data transmitting unit 164 areprovided. Duplicate explanation will be omitted as appropriate.

The information processing unit 151 acquires sensor data includingcaptured images and the internal state from the sensor 11 provided inthe unmanned aircraft 1 and various devices.

The display unit 161 causes the display 3A to display the compositeimage supplied from the image synthesis unit 39.

The input acquisition unit 162 outputs instruction informationrepresenting the content of the user's operation performed on thedisplay 3A having a touch panel to the control command generation unit163.

The control command generation unit 163 is supplied with theself-position and motion estimation result from the self-position andmotion estimation unit 36, and the failure and fall determinationinformation from the failure and fall determination unit 38. Further, acomposite image is supplied from the image synthesis unit 39 to thecontrol command generation unit 163.

The control command generation unit 163 determines whether there is afailure on the basis of the failure and fall determination information,and sets the operation mode of the smartphone 3. If a failure hasoccurred in the unmanned aircraft 1, the operation mode is set to thefalling mode, and if no failure has occurred, the operation mode is setto the in-flight mode.

The control command generation unit 163 generates a control commandrepresenting the direction instructed by the user according to theinstruction information supplied from the input acquisition unit 162.When the operation mode is the in-flight mode, the instructioninformation supplied from the input acquisition unit 162 is used as itis.

On the other hand, when the operation mode is the falling mode, thecontrol command generation unit 163 converts the instruction informationon the composite image coordinate system supplied from the inputacquisition unit 162 into the aircraft coordinate system on the basis ofthe composite image to generate a control command.

The control command generation unit 163 outputs the self-position andmotion estimation result, the failure and fall determinationinformation, and the control command to the data transmitting unit 164.

The data transmitting unit 164 transmits the self-position and motionestimation result, the failure and fall determination information, andthe control command supplied from the control command generation unit163 to the unmanned aircraft 1.

As described above, a part of the configuration of the unmanned aircraft1 shown in FIG. 7 may be provided on the smartphone 3.

Example of Computer

The series of processes described above can be executed by hardware orsoftware. When a series of processes are executed by software, theprograms constituting the software are installed from a programrecording medium on a computer embedded in dedicated hardware, ageneral-purpose personal computer, or the like.

FIG. 31 is a block diagram illustrating a configuration example ofhardware of a computer that executes a program to perform theabove-described series of processing.

A central processing unit (CPU) 1001, a read-only memory (ROM) 10O2, anda random access memory (RAM) 1003 are connected to each other via a bus1004.

An input/output interface 1005 is further connected to the bus 1004. Aninput unit 1006 including a keyboard and a mouse and an output unit 1007including a display and a speaker are connected to the input/outputinterface 1005. A storage unit 1008 including a hard disk or anonvolatile memory, a communication unit 1009 including a networkinterface, a drive 1010 driving a removable medium 1011 are connected tothe input/output interface 1005.

In the computer that has such a configuration, for example, the CPU 1001loads a program stored in the storage unit 1008 to the RAM 1003 via theinput/output interface 1005 and the bus 1004 and executes the program toperform the above-described series of processing.

The program executed by the CPU 1001 is recorded on, for example, theremovable medium 1011 or is provided via a wired or wireless transfermedium such as a local area network, the Internet, a digital broadcastto be installed in the storage unit 1008.

The program executed by the computer may be a program that performsprocesses chronologically in the procedure described in the presentspecification or may be a program that performs a process at a necessarytiming such as in parallel or upon being called.

In the present specification, a system is a collection of a plurality ofconstituent elements (devices, modules (components), or the like) andall the constituent elements may be located or not located in the samecasing. Therefore, a plurality of devices housed in separate housingsand connected via a network, and one device in which a plurality ofmodules are housed in one housing are both systems.

Also, the advantageous effects described in the present specificationare merely exemplary and are not intended as limiting, and otheradvantageous effects may be obtained.

The embodiment of the present technology is not limited to theabove-described embodiments, and various modifications can be madewithout departing from the gist of the present technology.

For example, the present technology can be configured as cloud computingin which one function is shared and processed in common by a pluralityof devices via a network.

Further, the respective steps described in the above-described flowchartcan be executed by one device or in a shared manner by a plurality ofdevices.

Furthermore, in a case where a plurality of kinds of processing areincluded in a single step, the plurality of kinds of processing includedin the single step may be executed by one device or by a plurality ofdevices in a shared manner.

<Combination Examples of Configurations>

The present technology can be configured as follows.

(1) A control device including: a control unit that controls movement ofan unmanned aircraft during a fall according to a control commandgenerated for an image showing a falling position, captured by theunmanned aircraft.

(2) The control device according to (1), further including: a synthesisunit that synthesizes information representing the falling position withan image captured by an imaging device provided on the unmanned aircraftand generates a composite image used for operation by a user.

(3) The control device according to (2), further including: anestimation unit that estimates a flight state of the unmanned aircrafton the basis of sensor data output by a sensor provided on the unmannedaircraft; and a falling position estimation unit that estimates thefalling position on the basis of the flight state and external forceapplied to the unmanned aircraft.

(4) The control device according to (3), further including: adetermination unit that detecting a fall of the unmanned aircraft on thebasis of at least one of the flight state and an internal state of theunmanned aircraft.

(5) The control device according to (3) or (4), wherein the synthesisunit generates the composite image by projecting and converting an imagecaptured by the imaging device with respect to a plane representing theground on the basis of the flight state and parameters of the imagingdevice and synthesizing information representing the falling positionwith an image obtained by the projection and conversion.

(6) The control device according to any one of (2) to (5), wherein thesynthesis unit generates the composite image including informationrepresenting the falling position and information representing aposition directly below the unmanned aircraft.

(7) The control device according to any one of (1) to (6), wherein thecontrol unit controls the movement of the unmanned aircraft according toa direction or position on an image designated by a user.

(8) The control device according to (7), wherein the control unitconverts the direction designated by the user into a direction on acoordinate system of the unmanned aircraft and controls the movement ofthe unmanned aircraft.

(9) The control device according to (7), wherein the control unitcontrols the movement of the unmanned aircraft on the basis of adifference between the position designated by the user and the fallingposition.

(10) The control device according to any one of (2) to (9), furtherincluding: a detection unit that detects an object appearing in an imagecaptured by the imaging device, wherein the synthesis unit synthesizesobject information representing the object detected by the detectionunit with the composite image.

(11) The control device according to (10), further including: an actionplanning unit that plans an action of the unmanned aircraft for avoidingcontact with the object detected by the detection unit, wherein thecontrol unit controls the movement of the unmanned aircraft on the basisof the planned action and the operation by the user.

(12) The control device according to (11), wherein the control unitgives priority to the operation by the user to control the movement ofthe unmanned aircraft.

(13) A control method including: allowing a control device to controlmovement of an unmanned aircraft during a fall according to a controlcommand generated for an image showing a falling position, captured bythe unmanned aircraft.

(14) A program for causing a computer to execute: controlling movementof an unmanned aircraft during a fall according to a control commandgenerated for an image showing a falling position, captured by theunmanned aircraft.

(15) An unmanned aircraft including: an imaging unit that captures asurrounding situation; and a control unit that controls movement of anunmanned aircraft during a fall according to a control command generatedfor an image showing a falling position, captured by the imaging unit.

(16) An information processing device including: a display control unitthat displays an image showing a falling position, captured by anunmanned aircraft; a generation unit that generates a control commandused for controlling the movement of the unmanned aircraft with respectto the image; and a transmitting unit that transmits the control commandto the unmanned aircraft.

(17) An information processing method for allowing an informationprocessing device to execute: displaying an image showing a fallingposition, captured by an unmanned aircraft; generating a control commandused for controlling the movement of the unmanned aircraft with respectto the image; and transmitting the control command to the unmannedaircraft.

(18) A program for causing a computer to execute: displaying an imageshowing a falling position, captured by an unmanned aircraft; generatinga control command used for controlling the movement of the unmannedaircraft with respect to the image; and transmitting the control commandto the unmanned aircraft.

REFERENCE SIGNS LIST

1 Unmanned aircraft

2 Controller

3 Smartphone

11 Sensor

12 Information processing unit

31 Wind speed vector acquisition unit

32 Image acquisition unit

33 Location information acquisition unit

34 IMU information acquisition unit

35 Internal information acquisition unit

36 Self-position and motion estimation unit

37 Falling position estimation unit

38 Failure and fall determination unit

39 Image synthesis unit

40 Data transmitting unit

41 Data receiving unit

42 Aircraft control unit

101 Object detection unit

111 Avoidance action generation unit

151 Information processing unit

161 Display unit

162 Input acquisition unit

163 Control command generation unit

164 Data transmission unit

1. A control device comprising: a control unit that controls movement ofan unmanned aircraft during a fall according to a control commandgenerated for an image showing a falling position, captured by theunmanned aircraft.
 2. The control device according to claim 1, furthercomprising: a synthesis unit that synthesizes information representingthe falling position with an image captured by an imaging deviceprovided on the unmanned aircraft and generates a composite image usedfor operation by a user.
 3. The control device according to claim 2,further comprising: an estimation unit that estimates a flight state ofthe unmanned aircraft on the basis of sensor data output by a sensorprovided on the unmanned aircraft; and a falling position estimationunit that estimates the falling position on the basis of the flightstate and external force applied to the unmanned aircraft.
 4. Thecontrol device according to claim 3, further comprising: a determinationunit that detecting a fall of the unmanned aircraft on the basis of atleast one of the flight state and an internal state of the unmannedaircraft.
 5. The control device according to claim 3, wherein thesynthesis unit generates the composite image by projecting andconverting an image captured by the imaging device with respect to aplane representing the ground on the basis of the flight state andparameters of the imaging device and synthesizing informationrepresenting the falling position with an image obtained by theprojection and conversion.
 6. The control device according to claim 5,wherein the synthesis unit generates the composite image includinginformation representing the falling position and informationrepresenting a position directly below the unmanned aircraft.
 7. Thecontrol device according to claim 1, wherein the control unit controlsthe movement of the unmanned aircraft according to a direction orposition on an image designated by a user.
 8. The control deviceaccording to claim 7, wherein the control unit converts the directiondesignated by the user into a direction on a coordinate system of theunmanned aircraft and controls the movement of the unmanned aircraft. 9.The control device according to claim 7, wherein the control unitcontrols the movement of the unmanned aircraft on the basis of adifference between the position designated by the user and the fallingposition.
 10. The control device according to claim 2, furthercomprising: a detection unit that detects an object appearing in animage captured by the imaging device, wherein the synthesis unitsynthesizes object information representing the object detected by thedetection unit with the composite image.
 11. The control deviceaccording to claim 10, further comprising: an action planning unit thatplans an action of the unmanned aircraft for avoiding contact with theobject detected by the detection unit, wherein the control unit controlsthe movement of the unmanned aircraft on the basis of the planned actionand the operation by the user.
 12. The control device according to claim11, wherein the control unit gives priority to the operation by the userto control the movement of the unmanned aircraft.
 13. A control methodcomprising: allowing a control device to control movement of an unmannedaircraft during a fall according to a control command generated for animage showing a falling position, captured by the unmanned aircraft. 14.A program for causing a computer to execute: controlling movement of anunmanned aircraft during a fall according to a control command generatedfor an image showing a falling position, captured by the unmannedaircraft.
 15. An unmanned aircraft comprising: an imaging unit thatcaptures a surrounding situation; and a control unit that controlsmovement of an unmanned aircraft during a fall according to a controlcommand generated for an image showing a falling position, captured bythe imaging unit.
 16. An information processing device comprising: adisplay control unit that displays an image showing a falling position,captured by an unmanned aircraft; a generation unit that generates acontrol command used for controlling the movement of the unmannedaircraft with respect to the image; and a transmitting unit thattransmits the control command to the unmanned aircraft.
 17. Aninformation processing method for allowing an information processingdevice to execute: displaying an image showing a falling position,captured by an unmanned aircraft; generating a control command used forcontrolling the movement of the unmanned aircraft with respect to theimage; and transmitting the control command to the unmanned aircraft.18. A program for causing a computer to execute: displaying an imageshowing a falling position, captured by an unmanned aircraft; generatinga control command used for controlling the movement of the unmannedaircraft with respect to the image; and transmitting the control commandto the unmanned aircraft.